Note These instructions assume you've read the following two sections of the main tutorial: Compiling C Programs on HPC (or the equivalent C++, Fortran 77 or Fortran 90 page) HPC Parallel Job Submission

Table of Contents

Compiling and Running

Compiling and running Parallel Hello World with MVAPICH is nearly the same as with OpenMPI. First, you will need to use switcher to change to MVAPICH:

switcher mpi = pgi-mvapich-1.1.0

If you want, you can use the GCC version of mvapich instead. The instructions in this tutorial will still be correct:

switcher mpi = gcc-mvapich-1.1.0

Make sure you log out and log back in so that switcher will update your MPI settings. After you've changed your compiler+MPI combination, recompile hello_parallel using the exact same command you used before. Depending on the language you're using, it is one of these commands:

• For Fortran 77: mpif77 hello_parallel.f -o hello_parallel
• For Fortran 90: mpif90 hello_parallel.f90 -o hello_parallel
• For C++: mpicxx hello_parallel.cc -o hello_parallel
• For C: mpicc hello_parallel.c -o hello_parallel

Running MVAPICH is very similar to running OpenMPI:

#!/bin/bash
: The above line tells Linux to use the shell /bin/bash to execute
: this script.  That must be the first line in the script.

: You must have no lines beginning with # before these
: PBS lines other than the /bin/bash line
#PBS -N 'hello_parallel'
#PBS -o 'qsub.out'
#PBS -e 'qsub.err'
#PBS -W umask=007
#PBS -q low_priority
#PBS -l nodes=5:ppn=4
#PBS -m bea

: Change our current working directory to the directory from which you ran qsub:
cd $PBS_O_WORKDIR

: Run the mpi program.  Notice the single dashes before machinefile and np:
mpirun -machinefile $PBS_NODEFILE -np 20 ./hello_parallel

Note that, for MVAPICH, your mpirun command uses single dashes (-np and -machinefile) instead of double dashes ("--").

Your job's output, and the process of linking, submitting, monitoring or canceling your job should be exactly the same as with the other MPI implementations.

MVAPICH: Issues and Advanced Usage

Less than Four Processes Per Machine

Usually you should use four processes per machine as we have done in this tutorial. Eventually, you might want to run less than four processes per machine (such as if you decide to use OpenMP). It isn't as simple as requesting fewer nodes in your -np option. Look here for instructions.

Environment Variables

Unlike the other MPI implementations, MVAPICH automatically forwards all environment variables to any processes it spawns (ie. hello_parallel). You nearly always want your environment variables to be forwarded, so that is a good thing. On the rare occasions when you will have to unset any environment variables that you don't want forwarded (via bash's unset or through the use of the env program). Don't worry if you don't know what that means – you will probably never need to prevent the forwarding of an environment variable.